Augmented reality techniques for simultaneously learning multiple languages

ABSTRACT

A system and method for using augmented reality for assisting speech development of multiple languages includes analyzing speech recorded in an environment to detect a word used in a conversation between a first speaker and a second speaker that is in a language different from other words in the conversation, wherein a target user is in the environment with the first speaker and the second speaker, isolating an object associated with the word within an augmented reality environment of the target user located in the environment, determining a confusion level of the target user based on a use of the word in the conversation, and implementing, by the processor, an augmented reality technique based on the confusion level of the target user.

TECHNICAL FIELD

The present invention relates to systems and methods for augmentedreality techniques for developing speech, and more specifically theembodiments of an augmented reality system for using augmented realityto assist with speech development of individuals simultaneously learningmultiple languages.

BACKGROUND

While learning to speak different languages simultaneously, it can beconfusing when a conversation switches between languages.

SUMMARY

An embodiment of the present invention elates to a method, andassociated computer system and computer program product for usingaugmented reality to assist with speech development of individualssimultaneously learning multiple languages. A processor of a computingsystem analyzes speech recorded in an environment to detect a word usedin a conversation between a first speaker and a second speaker that isin a language different from other words in the conversation, wherein atarget user is in the environment with the first speaker and the secondspeaker. An object associated with the word is isolated within anaugmented reality environment of the target user located in theenvironment. A confusion level of the target user is determined based ona use of the word in the conversation. An augmented reality technique isimplemented based on the confusion level of the target user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a block diagram of an augmented reality system, inaccordance with embodiments of the present invention.

FIG. 2 depicts an environment shared by two speakers and a target user,in accordance with embodiments of the present invention

FIG. 3 depicts the environment of FIG. 2, in which objects have beenidentified in the environment, in accordance with embodiments of thepresent invention.

FIG. 4 depicts the environment of FIG. 3, in which an object isisolated, in accordance with embodiments of the present invention.

FIG. 5 depicts implementation of a first augmented reality technique, inaccordance with embodiments of the present invention.

FIG. 6 depicts implementation of a second augmented reality technique,in accordance with embodiments of the present invention.

FIG. 7 depicts the environment of FIG. 2, in which an object has beengenerated, in accordance with embodiments of the present invention.

FIG. 8 depicts another implementation of the first augmented realitytechnique, in accordance with embodiments of the present invention.

FIG. 9 depicts another implementation of the second augmented realitytechnique, in accordance with embodiments of the present invention.

FIG. 10 depicts a flow chart of a method for using augmented reality forassisting speech development of multiple languages, in accordance withembodiments of the present invention.

FIG. 11 depicts a detailed flow chart for using augmented reality f©rassisting speech development of multiple languages, in accordance withembodiments of the present invention.

FIG. 12 depicts a block diagram of a computer system for an augmentedreality system of FIGS. 1-9, capable of implementing a method for usingaugmented reality for assisting speech development of multiple languagesof FIGS. 10-11, in accordance with embodiments of the present invention.

FIG. 13 depicts a cloud computing environment, in accordance withembodiments of the present invention.

FIG. 14 depicts abstraction model layers, in accordance with embodimentsof the present invention.

DETAILED DESCRIPTION

Some people are exposed to multiple languages. As a result, the sameobject may be referred differently in different languages at home,daycare, school etc, or by different individuals around the person.There are conflicting theories of people being exposed to multiplelanguages while learning to speak. One theory suggests that people havemore ability to pick up multiple languages at the age from 1-4 whenpeople generally learn to speak meaningfully. Another theory suggeststhat multiple languages when spoken in front of people who are learningto speak can confuse and burden the person, thus delaying proper speechdevelopment. Yet another theory suggests that when same object orscenario or task is interpreted using different words in differentlanguages, it can confuse a person who is unable to distinguish betweenlanguages. Confusion arising while learning to speak different languagesat the same time is alleviated by embodiments of the present invention,which uses artificial intelligence and augmented reality for automatedand optimized decisions for speech development specific to a person andthe person's environment in a multi-language environment.

Referring to the drawings, FIG. 1 depicts a block diagram of anaugmented reality system 100, in accordance with embodiments of thepresent invention. The augmented reality system 100 is a system forusing augmented reality for assisting speech development of multiplelanguages. The augmented reality system 100 may be useful for scenarioswhere conversations may include words in two or more different languagein the presence of a target user (e.g. child, student, etc.) that islearning to speak and/or learning a new language. Embodiments of theaugmented reality system 100 may be alternatively referred to a speechdevelopment tool, a language learning system, a mixed language therapysystem with augmented reality techniques, and the like.

In brief overview, embodiments of the augmented reality system 100involves a cognitive system listening for a scenario where mixed speechis used in a conversation between two speakers, which might confuse atarget user who is learning to speak. The target user is equipped withor others operates an augmented reality (AR) device (e.g. eye glasses)and wears the AR device or uses an application with AR capabilities onthe target user's mobile device. Alternatively, the speakers, such as aparent, guardian, or teacher, is using a device with the AR applicationor wearing AR glasses. Once the augmented reality system 100 detects amixed language condition (e.g. “Hey, look at that perro walking down thestreet”), the system uses augmented reality to identify the object (inthis case the dog walking down the street) and show multiple words thatcan define that person or object along with the words language origin.The use of augmented reality will help the target user understand thatthe word, in this case “perro”, isn't the English version of the wordbut in fact the Spanish version of “dog”. The augmented reality system100 clarifies for the target user that that there are two ways to saythe same thing and which word matches the rest of the conversation.Further, input mechanisms, such as IoT sensors, microphone, camera etc.connected to the augmented reality system 100 can analyze theconversation from the surrounding, and if the system identifies thatpeople are speaking in multiple languages, which might confuse thetarget user, then AI and AR systems will dynamically create gamingobject movement, comparative level of interest of the target user on thegame with respect to the conversation. Accordingly, based on themovement of the gaming object, the system 100 involves the target userto play the game and move away from the conversation environment, andusing cognitive computing the target user's attention will be divertedfrom the multi-language conversation topic. The augmented reality system100 analyzes a loudness of the multi-language conversation, andaccordingly dynamically identifies the new position of the target user,so that the target user does not listen to the multi-languageconversation.

The augmented reality sharing system 100 includes a computing system120. Embodiments of the computing system 120 include a computer system,a computer, a server, one or more servers, a backend computing system,and the like.

Furthermore, the augmented reality system 100 includes input mechanism110, input mechanism 111, an augmented reality device 112, and a profiledatabase 113 that are communicatively coupled to the computing system120 over a network 107. For instance, information/data is transmitted toand/or received from the input mechanism 110, the input mechanism 111,the augmented reality device 112, and the profile database 113 over anetwork 107. In an exemplary embodiment, the network 107 is a cloudcomputing network. Further embodiments of network 107 refer to a groupof two or more computer systems linked together. Network 107 includesany type of computer network known by individuals skilled in the art.Examples of network 107 include a LAN, WAN, campus area networks (CAN),home area networks (HAN), metropolitan area networks (MAN), anenterprise network, cloud computing network (either physical or virtual)e.g. the Internet, a cellular communication network such as GSM or CDMAnetwork or a mobile communications data network. In one embodiment, thearchitecture of the network 107 is a peer-to-peer, wherein in anotherembodiment, the network 107 is organized as a client/serverarchitecture.

In an exemplary embodiment, the network 107 further comprises, inaddition to the computing system 120, a connection to one or morenetwork-accessible knowledge bases 114, which are network repositoriescontaining information of the user preferences, selected or preferredlanguages, user's learning history, user performance, user studyhistory, user activity, user predefined rules, specific languagetranslation rules, user location, etc., network repositories or othersystems connected to the network 107 that are considered nodes of thenetwork 107. In an embodiment where the computing system 120 or networkrepositories allocate resources to be used by the other nodes of thenetwork 107, the computing system 120 and network-accessible knowledgebases 114 is referred to as servers.

The network-accessible knowledge bases 114 is a data collection area onthe network 107 which backs up and save all the data transmitted backand forth between the nodes of the network 107. For example, the networkrepository is a data center saving and cataloging the user preferences,selected or preferred languages, user's learning history, userperformance, user study history, user activity, user predefined rules,specific language translation rules, user location, etc., and the like,to generate both historical and predictive reports regarding aparticular user or a particular user augmented reality language and/orspeech development, etc. In an exemplary embodiment, a data collectioncenter housing the network-accessible knowledge bases 114 includes ananalytic module capable of analyzing each piece of data being stored bythe network-accessible knowledge bases 114. Further, the computingsystem 120 can be integrated with or as a part of the data collectioncenter housing the network-accessible knowledge bases 114. In analternative embodiment, the network-accessible knowledge bases 114 are alocal repository that is connected to the computing system 120.

The sensors 110, 111 are a sensor, an input device, or any inputmechanism. For example, sensor 110 and sensor 111 may be a combinationof a biometric sensor, a wearable sensor, an environmental sensor, acamera, a camcorder, a microphone, a peripheral device, a computingdevice, a mobile computing device, such as a smartphone or tablet,facial recognition sensor, voice capture device, and the like. Thesensors 110, 111 collect audio data, such as speech, from anenvironment. To collect the audio from the environment, the sensors 110,111 are positioned within an environment shared by the target user andthe speakers, worn by the speakers or target user, or otherwise disposedin a location that can result in obtaining voice data from aconversation. The collection of audio from a conversation is only donewith the knowledge and consent of the speakers and the target user, andcan be turned off at any time, thereby respecting the privacy of thespeakers and the target user.

In an exemplary embodiment, sensor 110 is a microphone of a smart mobilephone of target user or speaker and sensor 110 is a camera positionedwithin the environment, such as a car or living room. Sensors 110, 110can be two cameras that capture video data of a conversation, includingvoices, a posture, facial expressions, perspiration, muscle activity,gestures, etc. Sensors 110, 111 can be two microphones positioned nearbythe speakers and the target user to collect audio relating to aconversation between speakers and a target user. While FIG. 1 depictsone sensor 110 and one sensor 111, the system 100 may include more thantwo sensors and different combinations of sensors. Further sensors 110,111 that can be used with system 100 include other hardware inputdevices, such as an audio conversion device, digital camera orcamcorder, voice recognition devices, graphics tablet, a webcam, VRequipment, and the like. Sensors 110, 111 also can include a mobilecomputing device, such as a smartphone or tablet device, which may runvarious applications that contain data about the target user and/orspeakers. For example, a target user's smartphone may include a languagelearning application that may send progress data to the computing system120, or may send relevant learned or mastered words to the computingsystem 120. The mobile computing device as used as sensor may alsoutilize the device's camera, microphone, and other embedded sensors tosend information to the computing system 120. Moreover, embodiments ofsensors 110 may encompass other input mechanisms, such as a usercomputer that may send information to the computing system 120, whereinthe user computer may be loaded with software programs that are designedto track a productivity or work output level.

The augmented reality device 112 is an AR device that can be used by afirst user for displaying AR in the target user's environment. Theaugmented reality device 112 is configured to be worn by the targetuser. In an exemplary embodiment, the augmented reality device 112 is asmart contact lens configured to be won directly in the eye of the firstuser, or eyeglasses worn on the head of the target user. In anotherexemplary embodiment, the AR device 112 is a mobile computing device ofthe target user 112 that includes one or more software or hardwarecapabilities to support AR. The AR device 112 supports AR and hasnecessary components for communicating with the computing system 120 andoptionally the mobile devices of the target user and/or speakers.Furthermore, the AR device 112 generates an augmented reality experiencefor the user depending on the augmented reality technique determined bythe computing system 120. For instance, a generation module of the ARdevice 112 and/or the computing system 120 generates, renders, creates,etc. the augmented reality experience within an environment containingthe target user, as viewed by the target user using the AR device 112.The augmented reality experience is an interactive and immersivereal-world environment that includes computer-generated perceptualinformation generated by the generation module so that the real-worldenvironment is digitally manipulatable. The generation module iscommunicatively coupled to one or more sensors of the AR device 112and/or the target user mobile device, such as a GPS sensor, digitalcamera, accelerometer, gyroscopes, solid state compasses, RFID, andoptical sensors. The sensors of the AR device 112 and the target usermobile device provide location-based information, environmental details,mapping information, image data (e.g. from camera(s)), etc. which isused by the generation module in rendering the augmented realityexperience within the environment. Additionally, the generating moduleutilizes environmental sensors 110, 111 that may be located within theenvironment, such as beacon devices for micro-location data, cameras.

Referring still to FIG. 1, the profile database 113 is a database,storage medium, blockchain database, and the like, that stores profileinformation of the target user and profiles of other users. The profiledatabase 113 provides a cognitive storage ability for the history of theusers.

FIG. 2 depicts an environment 207 shared by two speakers and a targetuser, in accordance with embodiments of the present invention. Theenvironment 207 is a physical location in reality. The environment 207is defined by one or more boundaries. The one or more boundariesdefining the environment 207 can be a walls, surfaces, or otherstructures. Exemplary embodiments of environment 207 include one or morerooms of a house, a car, a floor of a building, an office, a kitchen ofa restaurant, a classroom of a school. In the example shown in FIG. 2,the environment 207 includes a first space 205 and a second space 206separated by a wall or similar partition. Moreover, the environment 207can be a dynamic environment. The dynamic environment moves with the ARdevice 112. Exemplary embodiments of a dynamic environment include aphysical area surrounding the AR device 112 that changes based on thephysical location of the AR device 112 as the target user moves. Thesize of the surrounding area depends on a specific, set physicaldistance (e.g. 20 foot radius from the AR device 112). The AR device 112is configured to communicate with various environmental sensors and/ormicro-location devices (e.g. beacon devices) for improved locationaccuracy for indoor environments or large event structures, such asstadiums, arenas, etc.

Inside the environment 207, a first speaker 201 and a second speaker 202are having a conversation. The target user 203 is also in theenvironment and can hear the conversation. The target user is equippedwith an AR device 112, such as AR glasses worn on the target user'shead. A plurality of sensors 110, 111 are disposed throughout theenvironment. The sensors 110, 111 include a microphone, a camera, asmart speaker, and a mobile phone of one of the speakers. In thisexample, the first speaker 201, the second speaker 202, and the targetuser 203 are in the same space 205 of the environment 207, and thesensors 110, 111, 111 are capturing the audio from a conversationbetween the first speaker 201 and the second speaker 202, andpotentially with the target user 203. If the first speaker 201 speaks ina first language to the second speaker 202 during the conversation, butone word of the conversation is spoken in a different language, thetarget user may be confused. The confusion of the target user can be ahindrance to the target user's 203 development of the language. Thesystem 100 determines whether the target user is confused and/or a levelor degree of confusion of the target user 203, and takes action usingaugmented reality, as described in greater detail infra.

Referring back to FIG. 1, the computing system 120 of the augmentedreality system 100 is equipped with a memory device 142 which storesvarious data/information/code, and a processor 141 for implementing thetasks associated with the augmented reality system 100. An augmentedreality learning assistance application 130 is loaded in the memorydevice 142 of the computing system 120. The augmented reality learningassistance application 130 can be an interface, an application, aprogram, a module, or a combination of modules. In an exemplaryembodiment, the augmented reality learning assistance application 130 isa software application running on one or more back end servers (e.g.computing system 120), servicing the augmented reality device 112 andpotentially a linked personal user computing device, and potentially amobile device associated with speakers surrounding the target user. Inother embodiments, the augmented reality learning assistance application130 is a software application running on the target user's mobile devicecontrolling the augmented reality device 112. In further embodiments,aspects of the augmented reality learning assistance application 130 areloaded onto the computing system 120 and the target user mobile device.

The augmented reality learning assistance application 130 of thecomputing system 120 includes a speech analysis module 131, an isolationmodule 132, a determining module 133, and an implementation module 134.A “module” refers to a hardware-based module, a software-based module,or a module that is a combination of hardware and software.Hardware-based modules include self-contained components such aschipsets, specialized circuitry and one or more memory devices, while asoftware-based module is a part of a program code or linked to theprogram code containing specific programmed instructions, which isloaded in the memory device of the computing system 120. A module(whether hardware, software, or a combination thereof) is designed toimplement or execute one or more particular functions or routines.

The speech analysis module 131 includes one or more components ofhardware and/or software program code for analyzing speech recorded inan environment to detect a word used in a conversation between a firstspeaker and a second speaker that is in a language different from otherwords in the conversation. For instance, sensors 110, 111 capture,collect, record, etc. audio spoken by one or more speakers having aconversation proximate a target user. The speech analysis module 131converts an audio file received from the sensors 110, 111 and/or mobilephones of the speakers or target users into text. The audio file is sentto the computing system 120, in response to collecting the audio from areal-world environment. As the speakers speak, one or more sensors 110,111 (e.g. microphones) collect the audio and creates a digital audiofile (e.g. creates a .mp3 pack) for transmitting to the computing system120 over the network 107. The digital file may be packetized and sentover the network in real-time to increase the speed in which thecomputing system 120 receives and parses the audio file over the network107. The one or more microphones, when powered on, can continuouslylisten for and collect audio of the conversation to listen for words ina different language than previous words (e.g. mixed languagecondition).

The speech analysis module 131 converts that audio file into text, andin response, parses the text to determine that a word used in a sentenceis in a different language than the previous words in the sentence andthe words after sentence. For example, the speech analysis module 131uses speech-to-text software to convert the audio file into text. Thelanguage spoken b user can also be detected using the speech-to-textsoftware as well a translation API associated with a translator. Thetranslator is a software application, engine, or tool for providingmachine translations for words between languages. The computing system120 may utilize the translator to translate the words into severallanguages. In an exemplary embodiment, the translator is a remoteservice accessed by the computing system 120 over network 107; however,a translation software application can be installed directly onto thecomputing system 120.

In response to converting the received audio file to text, the speechanalysis module 132 parses the text using natural language processingtechniques or comparable techniques to understand what words are beingused in the conversation and which language the words are being spokenin. The parsing of the text can include detecting a change in languagethat can initiate a particular response by the computing system 120. Thespeech analysis module 131 can continuously parse the text as the audiofile data is continuously received in real-time from the sensors 110,111. Based on the natural language analysis, the speech analysis module131 can determine that a word in a sentence is a different language thanthe previous words and potentially the words after.

The isolation module 132 includes one or more components of hardwareand/or software program code for isolating an object associated with theword (i.e. word identified to be in a different language) within anaugmented reality environment of the target user located in theenvironment. For example, the isolation module 132 searches for anobject in the environment 207 that correlates to the word detected as afunction of the analyzing. The object is searched by the isolationmodule 132 by processing environmental data received from a plurality ofsensors 110, 111 installed in the environment 111. In an exemplaryembodiment, an image recognition engine is used to locate and identifyobjects in the environment, based on data provided by a camera sensorlocated in the environment or the camera associated with the AR device112. If the object is physically present in the environment, theisolation module 132 alerts the target user to a location of the objectwithin the environment. Alerting the target user to the object mayinclude rendering an indicator in the augmented reality environmentshowing the target user where the object is, or may include rendering anoutline superimposed over the object so that the target user knowsprecisely which object the word is associated with. If the object is notphysically present in the environment, the isolation module 132generates the object in an augmented reality environment/experience ofthe target user that correlates to the word detected as a function ofthe analyzing. Accordingly, the isolation module 132 isolates the objectthat correlates to the mixed language word identified by the speechanalysis module 131; the isolation performed by the isolation module 132is output as a rendering by the AR device 112 of the target user, suchthat only the target user using the AR device 112 can visualize therendering that isolates the object.

The isolation module 132 optionally pre-captures the environment usingthe sensors 110, 111 and/or the AR device 112. For instance, the sensors110, 111 and/or the AR device 112 gathers and registers information fromthe surroundings that include the spoken content, people present,surrounding navigation paths (e.g. hallways, doors, etc.). The isolationmodule 132 may continuously process the environmental data as receivedby the sensors 110, 111 and/or the AR device 112. Pre-capturing andidentifying each object in the environment can speed up the isolation ofthe object to keep up with the conversation.

Referring still to FIG. 1, the determining module 133 includes one ormore components of hardware and/or software program code for determininga confusion level of the target user based on a use of the word in theconversation. For instance, the determination module 133 determineswhether the target user is confused when the word is spoken inconversation. The confusion level is based on the user's reaction tohearing the word and the user's reaction to seeing the object associatedwith the word in the augmented reality environment. The target user'sreactions are captured by sensors 110, 111 and/or by cameras, sensors,microphones, etc. of the AR device 112. For example, the AR device 112camera may detect the facial expression of the target user and concludethat the user is confused by the use of the word, with a degree ofconfidence. The AR device 112 camera may later detect that the confusedfacial expression based on hearing the word has changed to an interestedfacial expression based on seeing the object in the environment. Thechange in facial expressions can affect the confusion level of thetarget user, impacting which augmented reality technique to implement inthe moment. Further, sensors of the AR device 112, such as anaccelerometer or gyroscope, can provide data regarding a movement of abody part of the target user that suggests confusion, such as a tilt ofthe head (if the target user AR device is being worn on the head), or amovement of the hand to head (if the target user AR device is being wornon a wrist).

Further factors are considered to determine a confusion level of theuser, such as mobility behavior of the target user and biometric data ofthe target user. The mobility behavior of the target user is analyzed todetermine whether the user is confused or interested in theconversation. For example, if the target user is walking away from theconversation or turning away from the isolated object, the determinationmodule 133 concludes that the user is confused, which could have harmfuleffects to the development of speech or language learning. Biometricdata can be used to determine a level of confusion, such as a heartrate, a perspiration content, accelerometer data indicative of gesturesof confusion, such as hand being raised to the head.

The determining module 133 optionally applies a scheme to quantify aconfusion level of the user into a numerical score. The numerical scorecan be weighted by confidence scores, and can include a plurality offactors that contribute to the confusion of the user. The score isdetermined by data collected by a plurality of sensors and input devicesto provide a plurality of target user metrics based on a plurality offactors related to the target user and the target user'sbehavior/actions. The metrics may be based on a historical learningsystem or real-time data relating to a person's individualcharacteristics, as compared with the profiles in the profiles database113. The profiles database 113 is accessed by the determination module133 as part of the calculation of the confusion score of the targetuser. The target user's speech is gathered, and development assessed andscored overtime w.r.t target user with similar profiles. The scoreoutput from the AI algorithm applying the profile data is utilized bythe determining module 133 for calculating the confusion score inspecific scenarios for a specific target user.

For each detection of the mixed language word, the determining module133 calculates a confusion score that can be applied to a pre-determinedthreshold. If the confusion score exceeds the threshold, then a firstaugmented reality technique is implemented. If the confusion score isbelow the threshold, then a second augmented reality technique isimplemented that is different than the first augmented realitytechnique.

The implementation module 134 includes one or more components ofhardware and/or software program code for implementing an augmentedreality technique based on the confusion level of the target user. Basedon the level of confusion level, the implementation module 134 predictswhether a real time translation will be preferable for the target useror moving the target user away the surrounding conversation will bepreferable. The implementation module 134 improves over time and is aself-learning module which assesses the improvement of the target user'sspeech and combining that factor with the confusion score. For example,if the target user is able to pick up multiple languages, real timetranslation will be weighted higher in view of the confusion score. Ifthe target user is struggling and getting confused with the same objectsbeing referred with multiple words from different languages and thespeech development is hampered, real time translation will be weightedlower in view of the confusion score. Accordingly, the implementationmodule 134 modifies the confusion score based on a deeper understandingof the target user's recent accomplishments and struggles with speechdevelopment and/or language learning; the data for analyzing theimprovements, struggles, accomplishments, etc. come from the profiledatabase 113 that is constantly updated over time. Alternatively, theimplementation module 134 determines which augmented reality techniqueto implement based solely on the confusion score.

If the target user's confusion level is low enough based on thepre-determine threshold, the implementation module 134 implements afirst augmented reality technique. The first augmented reality techniqueis designed to provide the target user with a translation of the wordbeing used in the conversation in combination with the isolation of theobject to advance the learning of the language. For example, the firstaugmented reality technique includes rendering a text translation of theword in the augmented reality of the target user, which assists thetarget user in learning the word in multiple languages. The AR device112 displays a text translation next to or near the object isolated inthe target user's environment so that the target user can be assisted tolearn the word in multiple languages. Because the confusion level of thetarget user is examined, the risk of overly confusing the target user bydisplaying more information pertaining to the mixed language word isavoided.

If the target user's confusion level exceeds the pre-determinedthreshold, the implementation module 134 implements a second augmentedreality technique that is different than the first augmented realitytechnique. The second augmented reality technique is designed todistract the target user from the conversation. For example, the secondaugmented reality technique includes initiating an augmented realitygame within the augmented reality of the target user to distract thetarget user from the conversation. The implementation module 134dynamically creates a gaming context within the augmented reality of thetarget user that navigates the target user away from the first speakerand the second speaker by controlling a movement of the target user witha game or object rendered by the AR device 112; a distance that thetarget user is navigated away from the conversation correlates to aloudness of the conversation. For example, the louder the conversation,the further away the game will take the user. The target user, byplaying the AR game, physically moves away from the conversation toavoid confusion or detrimental effects to the target user's speechdevelopment and/or language learning.

An exemplary embodiment of the system 100 will now be described withreference to FIGS. 3-9. FIG. 3 depicts the environment 207 of FIG. 2, inwhich objects have been identified within the environment, in accordancewith embodiments of the present invention. In the illustratedembodiment, objects 210 a and 210 b have been identified in theenvironment 207 in the manner described above. Object 210 a is atelevision and object 201 b is a table. FIG. 4 depicts the environment207 of FIG. 3, in which an object is isolated, in accordance withembodiments of the present invention. Object 210 a is isolated becausethe first speaker 201 and the second speaker 202 were discussingwatching a movie together. By way of example, the first speaker 201states, “Why don't we watch the new comic book movie on the living roomtelevisión.” In response to the detection that the word “televisión” isin a different language than the other words in the sentence, the systemisolates object 210 a in the augmented reality of the target user, shownas heavier line thickness in FIG. 4. The system analyzes the confusionlevel of the target user based on the user of the word “televisión” andthe reaction of seeing the object 210 a highlighted in the augmentedreality.

FIG. 5 depicts implementation of a first augmented reality technique, inaccordance with embodiments of the present invention. In the illustratedembodiment, the system has determined that the confusion level of thetarget user is below the threshold, and thus renders the translation inthe augmented reality of the target user near the object 210 a. Therendering of the translation into another language can assist the targetuser to learn the word for the isolated object in multiple languages.

FIG. 6 depicts implementation of a second augmented reality technique,in accordance with embodiments of the present invention. In theillustrated embodiment, the system has determined that the confusionlevel of the target user is above the threshold, and thus initiates adynamic AR-based game for the target user to play. The AR game generatesobject 225 that can move around the environment and draw the target useraway from the conversation between the speakers 201, 202. As shown inFIG, 6, the target user 203 is in space 205 of the environment 207. Thedynamic AR game controls the movement of the target user by enticing thetarget user to follow the object 225 into space 206, away from theconversation between the speakers 201, 202.

FIG. 7 depicts the environment 207 of FIG. 2, in which an object hasbeen generated, in accordance with embodiments of the present invention.In the illustrated embodiment, the first speaker 201 states, “We need toplace a new silla in the corner of our living room.” In response to thedetection that the word “silla” (chair) is in a different language thanthe other words in the sentence, the system searches for a chair in theenvironment. Because a chair is not located in the environment, thesystem generates object 210 c which is rendered as a chair in theaugmented reality environment of the target user 203, and isolatesobject 210 c, shown as heavier line thickness in FIG. 7. The systemanalyzes the confusion level of the target user based on the user of theword “silla” and the reaction of seeing the object 210 c highlighted inthe augmented reality.

FIG. 8 depicts another implementation of a first augmented realitytechnique, in accordance with embodiments of the present invention. Inthe illustrated embodiment, the system has determined that the confusionlevel of the target user is below the threshold, and thus renders thetranslation in the augmented reality of the target user near thegenerated object 210 c. The rendering of the translation into anotherlanguage can assist the target user to learn the word for the isolatedobject in multiple languages.

FIG. 9 depicts another implementation of the second augmented realitytechnique, in accordance with embodiments of the present invention. Inthe illustrated embodiment, the system has determined that the confusionlevel of the target user is above the threshold, and thus initiates adynamic AR-based game for the target user to play. The AR game generatesobject 225 that can move around the environment and draw the target useraway from the conversation between the speakers 201, 202. As shown inFIG. 9, the target user 203 is in space 205 of the environment 207. Thedynamic AR game controls the movement of the target user by enticing thetarget user to follow the object 225 into space 206, away from theconversation between the speakers 201, 202.

Various tasks and specific functions of the modules of the computingsystem 120 may be performed by additional modules, or may be combinedinto other modules(s) to reduce the number of modules. Further, anembodiment of the computer or computer system 120 comprises specialized,non-generic hardware and circuitry (i.e., specialized discretenon-generic analog, digital, and logic-based circuitry) (independentlyor in combination) particularized for executing only methods of thepresent invention. The specialized discrete non-generic analog, digital,and logic-based circuitry includes proprietary specially designedcomponents (e.g., a specialized integrated circuit, such as for examplean Application Specific Integrated Circuit (ASIC), designed for onlyimplementing methods of the present invention).

Furthermore, the augmented reality system 100 uses specific hardware,such as augmented reality devices (e.g. smart contact lenses,eyeglasses), for displaying AR environments and augmentations to the ARenvironment. The AR displayed by the augmented reality devices can beaugments according to specific set of rules that are uniquely applied toeach user. The augmented reality system 100 provides a technicalsolution by augmenting AR environments displayed by specialized devicesbased on a set of specific rules associated with a confusion level of auser.

Referring now to FIG. 10, which depicts a flow chart of a method 300 forusing augmented reality for assisting speech development of multiplelanguages, in accordance with embodiments of the present invention. Oneembodiment of a method 300 or algorithm that may be implemented forusing augmented reality for assisting speech development of multiplelanguages with the augmented reality system 100 described in FIGS. 1-9using one or more computer systems as defined generically in FIG. 12below, and more specifically by the specific embodiments of FIG. 1.

Embodiments of the method 300 for using augmented reality for assistingspeech development of multiple languages, in accordance with embodimentsof the present invention, may begin at step 301 wherein step 301analyzes speech from an environment to detects word(s) in differentlanguages than other words in the speech. Step 302 isolates an objectthat is associated with or correlates with the word in an ARenvironment. Step 303 determines a confusion level of the user. Step 304implements an AR technique based on the confusion level of the user.

FIG. 11 depicts a detailed flow chart of a method 400 for usingaugmented reality for assisting speech development of multiplelanguages, in accordance with embodiments of the present invention. Atstep 401, two or more speakers have a conversation in an environmentthat also includes a target user that is learning a language orundergoing speech development in more than one language. At step 402,the system detects a word from a sentence of the conversation that isspoken in another language from the other words in the sentence orconversation. Step 403 searches for an object in the environment thatcorrelates with the detected word spoken in the different language thanthe other words. Step 404 determines whether an object can be located inthe physical environment. If no, step 405 generates the object withinthe environment using augmented reality. If yes, then step 406highlights the physical object using augmented reality. Step 407assesses the user to determine a confusion level of the user. Step 408determines whether the confusion level or score exceeds a predeterminedthreshold, which can vary over time as the user improves. If no, thenstep 409 renders a translation in the AR environment using augmentedreality. If yes, then step 410 initiates an AR-based game to relocatethe user.

FIG. 12 depicts a block diagram of a computer system for the augmentedreality system 100 of FIGS. 1-9, capable of implementing methods forusing augmented reality for assisting speech development of multiplelanguages of FIGS. 10-11, in accordance with embodiments of the presentinvention. The computer system 500 may generally comprise a processor591, an input device 592 coupled to the processor 591, an output device593 coupled to the processor 591, and memory devices 594 and 595 eachcoupled to the processor 591. The input device 592, output device 593and memory devices 594, 595 may each be coupled to the processor 591 viaa bus. Processor 591 may perform computations and control the functionsof computer system 500, including executing instructions included in thecomputer code 597 for the tools and programs capable of implementing amethod for using augmented reality for assisting speech development ofmultiple languages in the manner prescribed by the embodiments of FIGS.10-11 using the augmented reality system 100 of FIGS. 1-9, wherein theinstructions of the computer code 597 may be executed by processor 591via memory device 595. The computer code 597 may include software orprogram instructions that may implement one or more algorithms forimplementing the method for using augmented reality for assisting speechdevelopment of multiple languages, as described in detail above. Theprocessor 591 executes the computer code 597. Processor 591 may includea single processing unit, or may be distributed across one or moreprocessing units in one or more locations (e.g., on a client andserver).

The memory device 594 may include input data 596. The input data 596includes any inputs required by the computer code 597. The output device593 displays output from the computer code 597. Either or both memorydevices 594 and 595 may be used as a computer usable storage medium (orprogram storage device) having a computer-readable program embodiedtherein and/or having other data stored therein, wherein thecomputer-readable program comprises the computer code 597. Generally, acomputer program product (or, alternatively, an article of manufacture)of the computer system 500 may comprise said computer usable storagemedium (or said program storage device).

Memory devices 594, 595 include any known computer-readable storagemedium, including those described in detail below. In one embodiment,cache memory elements of memory devices 594, 595 may provide temporarystorage of at least some program code (e.g., computer code 597) in orderto reduce the number of times code must be retrieved from bulk storagewhile instructions of the computer code 597 are executed. Moreover,similar to processor 591, memory devices 594, 595 may reside at a singlephysical location, including one or more types of data storage, or bedistributed across a plurality of physical systems in various forms.Further, memory devices 594, 595 can include data distributed across,for example, a local area network (LAN) or a wide area network (WAN).Further, memory devices 594, 595 may include an operating system (notshown) and may include other systems not shown in FIG. 12.

In some embodiments, the computer system 500 may further be coupled toan Input/output (I/O) interface and a computer data storage unit. An I/Ointerface may include any system for exchanging information to or froman input device 592 or output device 593. The input device 592 may be,inter alia, a keyboard, a mouse, etc. or in some embodiments thetouchscreen of a computing device. The output device 593 may be, interalia, a printer, a plotter a display device (such as a computer screen),a magnetic tape, a removable hard disk, a floppy disk, etc. The memorydevices 594 and 595 may be, inter alia, a hard disk, a floppy disk, amagnetic tape, an optical storage such as a compact disc (CD) or adigital video disc (MID), a dynamic random access memory (DRAM), aread-only memory (ROM), etc. The bus may provide a communication linkbetween each of the components in computer 500, and may include any typeof transmission link, including electrical, optical, wireless, etc.

An I/O interface may allow computer system 500 to store information(e.g., data or program instructions such as program code 597) on andretrieve the information from computer data storage unit (not shown).Computer data storage unit includes a known computer-readable storagemedium, which is described below. In one embodiment, computer datastorage unit may be a non-volatile data storage device, such as amagnetic disk drive (i.e., hard disk drive) or an optical disc drive(e.g., a CD-ROM drive which receives a CD-ROM disk). In otherembodiments, the data storage unit may include a knowledge base or datarepository 125 as shown in FIG. 1.

As will be appreciated by one skilled in the art, in a first embodiment,the present invention may be a method; in a second embodiment, thepresent invention may be a system; and in a third embodiment, thepresent invention may be a computer program product. Any of thecomponents of the embodiments of the present invention can be deployed,managed, serviced, etc. by a service provider that offers to deploy orintegrate computing infrastructure with respect to using augmentedreality for assisting speech development of multiple languages. Thus, anembodiment of the present invention discloses a process for supportingcomputer infrastructure, where the process includes providing at leastone support service for at least one of integrating, hosting,maintaining and deploying computer-readable code (e.g., program code597) in a computer system (e.g., computer system 500) including one ormore processor(s) 591, wherein the processor(s) carry out instructionscontained in the computer code 597 causing the computer system to useaugmented reality for assisting speech development of multiplelanguages. Another embodiment discloses a process for supportingcomputer infrastructure, where the process includes integratingcomputer-readable program code into a computer system 500 including aprocessor.

The step of integrating includes storing the program code in acomputer-readable storage device of the computer system 500 through useof the processor. The program code, upon being executed by theprocessor, implements a for using augmented reality for assisting speechdevelopment of multiple languages. Thus, the present invention disclosesa process for supporting, deploying and/or integrating computerinfrastructure, integrating, hosting, maintaining, and deployingcomputer-readable code into the computer system 500, wherein the code incombination with the computer system 500 is capable of performing amethod for using augmented reality for assisting speech development ofmultiple languages.

A computer program product of the present invention comprises one ormore computer-readable hardware storage devices having computer-readableprogram code stored therein, said program code containing instructionsexecutable by one or more processors of a computer system to implementthe methods of the present invention.

A computer system of the present invention comprises one or moreprocessors, one or more memories, and one or more computer-readablehardware storage devices, said one or more hardware storage devicescontaining program code executable by the one or more processors via theone or more memories to implement the methods of the present invention.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer-readable storagemedium (or media) having computer-readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer-readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer-readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer-readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer-readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer-readable program instructions described herein can bedownloaded to respective computing/processing devices from acomputer-readable storage medium or to an external computer or externalstorage device via a network, for example, the Internet, a local areanetwork, a wide area network and/or a wireless network. The network maycomprise copper transmission cables, optical transmission fibers,wireless transmission, routers, firewalls, switches, gateway computersand/or edge servers. A network adapter card or network interface in eachcomputing/processing device receives computer-readable programinstructions from the network and forwards the computer-readable programinstructions for storage in a computer-readable storage medium withinthe respective computing/processing device.

Computer-readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine-dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer-readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer-readable program instructions by utilizing state information ofthe computer-readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer-readable program instructions.

These computer-readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer-readable program instructionsmay also be stored in a computer-readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that thecomputer-readable storage medium having instructions stored thereincomprises an article of manufacture including instructions whichimplement aspects of the function/act specified in the flowchart and/orblock diagram block or blocks.

The computer-readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce acomputer-implemented process, such that the instructions which executeon the computer, other programmable apparatus, or other device implementthe functions/acts specified in the flowchart and/or block diagram blockor blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

It is to be understood that although this disclosure includes a detaileddescription on cloud computing, implementation of the teachings recitedherein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g., networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as Follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported, providing transparency for both theprovider and consumer of the utilized service.

Service Models are as Follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as Follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure that includes anetwork of interconnected nodes.

Referring now to FIG. 13, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 includes one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A, 54B,54C and 54N shown in FIG. 13 are intended to be illustrative only andthat computing nodes 10 and cloud computing environment 50 cancommunicate with any type of computerized device over any type ofnetwork and/or network addressable connection (e.g., using a webbrowser).

Referring now to FIG. 14, a set of functional abstraction layersprovided by cloud computing environment 50 (see FIG. 13) are shown. Itshould be understood in advance that the components, layers, andfunctions shown in FIG. 14 are intended to be illustrative only andembodiments of the invention are not limited thereto. As depicted, thefollowing layers and corresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may include applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provides pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and GUI and augmented reality 96.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein

1. A method comprising: analyzing, by a processor of a computing system, speech recorded in an environment to detect a word used in a conversation between a first speaker and a second speaker that is in a language different from other words in the conversation, wherein a target user is in the environment with the first speaker and the second speaker; isolating, by the processor, an object associated with the word within an augmented reality environment of the target user located in the environment; determining, by the processor, a confusion level of the target user based on a use of the word in the conversation; and implementing, by the processor, an augmented reality technique based on the confusion level of the target user.
 2. The method of claim 1, wherein the augmented reality technique includes: rendering, by the processor, a text translation of the word in the augmented reality of the target user, which assists the target user in learning the word in multiple languages.
 3. The method of claim 1, wherein the augmented reality technique includes: initiating, by the processor, an augmented reality game within the augmented reality of the target user to distract the target user from the conversation.
 4. The method of claim 3, wherein the initiating the augmented reality game includes: dynamically creating, by the processor, a gaming context within the augmented reality of the target user that navigates the target user away from the first speaker and the second speaker; wherein a distance that the target user is navigated away from the conversation correlates to a loudness of the conversation.
 5. The method of claim 1, wherein the isolating includes: searching, by the processor, for an object in the environment that correlates to the word detected as a function of the analyzing, by processing environmental data received from a plurality of input mechanisms installed in an environment; and alerting, by the processor, the target user to a location of the object within the environment.
 6. The method of claim 1, wherein the isolating includes: generating, by the processor, an object in the augmented reality environment of the target user that correlates to the word detected as a function of the analyzing.
 7. The method of claim 1, wherein the confusion level of the target user is predicted by detecting a mobility behavior and a facial expression of the target user in response to isolating the object associated with the word.
 8. (canceled)
 9. The computing system of claim 8, wherein the augmented reality technique includes: rendering, by the processor, a text translation of the word in the augmented reality of the target user, which assists the target user in learning the word in multiple languages.
 10. The computing system of claim 8, wherein the augmented reality technique includes: initiating, by the processor, an augmented reality game within the augmented reality of the target user to distract the target user from the conversation.
 11. The computing system of claim 10, wherein the initiating the augmented reality game includes: dynamically creating, by the processor, a gaming context within the augmented reality of the target user that navigates the target user away from the first speaker and the second speaker; wherein a distance that the target user is navigated away from the conversation correlates to a loudness of the conversation.
 12. The computing system of claim 8, wherein the isolating includes: searching, by the processor, for an object in the environment that correlates to the word detected as a function of the analyzing, by processing environmental data received from a plurality of input mechanisms installed in an environment; and alerting, by the processor, the target user to a location of the object within the environment.
 13. The computing system of claim 8, wherein the isolating includes: generating, by the processor, an object in the augmented reality environment of the target user that correlates to the word detected as a function of the analyzing.
 14. The computing system of claim 8, wherein the confusion level of the target user is predicted by detecting a mobility behavior and a facial expression of the target user in response to isolating the object associated with the word.
 15. The computing system of claim 8, further comprising: creating, by the processor, a profile of the target user over time, and comparing the profile of the target user to a database of profiles of other target users to assist in determining the confusion level of the target user.
 16. A computer program product, comprising a computer readable hardware storage device storing a computer readable program code, the computer readable program code comprising an algorithm that when executed by a computer processor of a computing system implements a method comprising: analyzing, by a processor of a computing system, speech recorded in an environment to detect a word used in a conversation between a first speaker and a second speaker that is in a language different from other words in the conversation, wherein a target user is in the environment with the first speaker and the second speaker; isolating, by the processor, an object associated with the word within an augmented reality environment of the target user located in the environment; determining, by the processor, a confusion level of the target user based on a use of the word in the conversation; and implementing, by the processor, an augmented reality technique based on the confusion level of the target user.
 17. The computer program product of claim 16, wherein the augmented reality technique includes: rendering, by the processor, a text translation of the word in the augmented reality of the target user, which assists the target user in learning the word in multiple languages.
 18. The computer program product of claim 16, wherein the augmented reality technique includes: initiating, by the processor, an augmented reality game within the augmented reality of the target user to distract the target user from the conversation.
 19. The computer program product of claim 18, wherein the initiating the augmented reality game includes: dynamically creating, by the processor, a gaming context within the augmented reality of the target user that navigates the target user away from the first speaker and the second speaker; wherein a distance that the target user is navigated away from the conversation correlates to a loudness of the conversation.
 20. The computer program product of claim 16, wherein the isolating includes: generating, by the processor, an object in the augmented reality environment of the target user that correlates to the word detected as a function of the analyzing. 